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Abstract. In a previous paper, we had proved that the permutation group 
generated by the round functions of an AES-hke cipher is primitive. Here we 
apply the O'Nan Scott classification of primitive groups to prove that this group 
is the alternating group. 



1. Introduction 

According to Shannon |Sha49l p. 657], a cipher "is defined abstractly as a set of 
transformations". Coppersmith and Grossman [CG75j . and later in 1988 Kaliski, 
Rivest and Sherman [KRS88j . called attention to the group generated by a cipher. 
One of the motivations for the work of Kaliski et al. is that at that time Triple 
DES was being suggested as an improvement to DES. This meant replacing the use 
of single DES transformation T^, where a is a key, with the composition TaTbT^, 
where a, b, c are three DES keys. If it was the case that the transformations of 
DES form a group, then Triple DES would have been of course no more than 
DES itself. More generally, Kaliski et al. showed that if the group generated by 
the transformations of a cipher is too small, then the cipher is exposed to certain 
cryptanalytic attacks. 

It was later proved by Wernsdorf |Wer93j that the group generated by the round 
functions of DES (which are even permutations) is the alternating group. This 
implies that the group generated by the DES transformations with independent 
subkeys is also the alternating group. (We are not aware of any work in this 
context that tries to take account of the key schedule.) 

Wernsdorf used ad hoc methods in |Wer02j to prove that the permutation group 
G generated by the round functions of AES is the alternating group. (Here, too, 
these functions are even permutations.) Sparr and Wernsdorf have recently given 
another, permutation group theoretic proof in [SW08] . 

The goal of this paper is to give a different proof of this fact, building upon 
our earlier paper [CDVS08] . There we had proved that the group G is primitive. 
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In the course of doing that we answered a question of Paterson jPat99] about the 
possibihty of embedding a trapdoor in a cipher by having the group generated by 
the cipher act imprimitively. 

In this paper we work under certain cryptographic assumptions (see Section ^ 
that are a stripped down, simphfied version of those of |CDVS08] . (These are 
also satisfied by AES.) We first give, for the convenience of the reader, a short 
group-theoretic version of the main result of [CDVSOS] under these assumptions. 
We then appeal to the O'Nan-Scott classification of primitive groups to prove 
that the group generated by the round functions of a cryptosystem satisfying our 
assumptions is the alternating group. 

We are very grateful to Ralph Wernsdorf for several useful suggestions. 

2. Preliminaries 

In the rest of the paper, we tend to adopt the notation of |DR02] . 
Let V = V{d, 2), the vector space of dimension d over the field GF(2) with two 
elements, be the state (or message) space. V has n = 2'^ elements. 
For any v & V, consider the translation by v, that is the map 

W ^ W + V. 

In particular, ctq is the identity map on V . The set 

T = {a,:veV} 

is an elementary abelian, regular subgroup of Sym(V). In fact, the map 

V -^T 

(2.1) 

V ^ ay 

is an isomorphism of the additive group V onto the multiplicative group T. 

We consider a key- alternating block cipher (see Section 2.4.2 of |DR02j ) which 
consists of a fixed number of iterations of a function of the form pcjfe, where k & V. 
Such a function is called a round function, and the parameter k is called the round 
key. (We write maps left-to-right, so p operates first.) Here p is a fixed permutation 
operating on the vector space V. Therefore each round consists of an application 
of p, followed by a key addition. This covers for instance AES with independent 
subkeys. Let G = { pau : k E V) he the group of permutations of V generated 
by the round functions. Choosing /c = we see that p E G, and thus T < G. It 
follows that G = {T, p) . 

We assume p = 7A, where 7 and A are permutations. Here 7 is a bricklayer 
transformation, consisting of a number of S-boxes. The message space V is written 
as a direct sum 

V = Vi®---®Vn,, 

rit > 1, where each Vi has the same dimension m > 1 over GF(2). As > 1, 
this implies that d = mnt is not a prime number. For v E V, we will write 
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V = vi + ■ ■ ■ + Vnt, where Vi E Vi. Also, we consider the projections iTi : V —>■ Vi, 
which map v ^ Vi. We have 

where the 7j are S-boxes, which we allow to be different for each V^. 

A is a linear function (usually called a linear mixing layer). The only assumption 
we will be making about A is Cryptographic Assumption below. 

In AES the S-boxes are all equal, and consist of inversion in the field GF(2^) 
with 2* elements (see later in this paragraph), followed by an affine transformation, 
that is, a linear transformation, followed by a translation. When interpreting AES 
in our scheme, we take advantage of the well-known possibility of moving the linear 
part of the affine transformation to the linear mixing layer, and incorporating the 
translation in the key addition (see for instance |MR02j ). Thus in our scheme for 
AES we have m = 8, we identify each Vi with GF(2^), and we take X'ji = 
so that 7j maps nonzero elements to their inverses, and zero to zero. As usual we 
will simply say that ji acts by inversion. 

We will work under the following 

Cryptographic Assumptions. Consider an AES-like cryptosystem as described 
above, which satisfies the following conditions. 

(1) O7 = and 7^ = 1, the identity transformation. 

(2) There is 1 < r < m/2 such that the following hold. 

(a) For all ^ V E Vi, the image of the map Vi Vi, which maps 
X I— i> (x + f )7j +X7j, has size greater than 2™-"''"^, and it is not a coset 
of a subspace. 

(b) There is no subspace of Vi, invariant under 7j, of codimension less 
than or equal to 2r. 

(3) There are no subspaces U, U', U" (except { } and V) that are the sum of 
some of the Vi, and such that UX = U' and U'X = U" . 

In |CDV S08] we have proved under certain abstract and general assumptions a 
result that specializes to the following: 

Theorem 1. Suppose a cryptosystem satisfies the Cryptographic Assumptions. 
Then the group G generated by its round functions acts primitively on the message 
space V . 

We give a short, group-theoretic proof of this in Section [3l This we do for the 
convenience of the reader, as we will need to refer to part of the proof in Section El 
We are grateful to the referee of another paper for this proof. 

In the rest of the paper we prove the following 

Theorem 2. Suppose a cryptosystem satisfies the Cryptographic Assumptions. 

Then the group G generated by its round functions is the alternating group 
A\t{V). 

The same holds for the group generated by the cryptosystem with independent 
subkeys. 
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A word about the parity of the group G is in order here. Over V = V{d,2), 
non-trivial translations are clearly involutions without fixed points, and thus even 
permutations. Also, for d > 2 the group GL((i, 2) = SL{d,2) is perfect, so that 
in particular it has no (normal) subgroup of order 2, and it is thus contained in 
A\t{V). 

We now show that 7 is also even, so that G < Alt(V"). In fact, 7 is the product 
of fit permutations gi, acting as 7^ on Vi, and as the identity on Vj, j i. This 
means that every 2-cycle in 7^ gives rise to 2'^"™ 2-cycles in Qi. Now the number 
2d-m jg gygj^^ as d — m = ritm — m > m, rit > 1 hj assumption, and m > 2 by 
Cryptographic Assumption ([1]). It follows that each Qi is even, and thus so is 7. 
(The same argument proves that 7 is even, even without assuming that it is an 
involution, as we do here.) 

Condition ([T]) is clearly satisfied by AES. As we said above, we take advantage 
here of the possibility of assuming that 7 is simply componentwise inversion. 

Condition ( l2al) is also well-known to be satisfied, with r = 1 (see |Nyb94| but 
also |DR06j ). as the image of that map has size 2'' — 1. 

As to Condition ( ]2bl) . it is also satisfied by AES with r = 1. For that, one could 
just use GAP |GAP05] to verify that the only nonzero subspaces of GF(2^) which 
are invariant under inversion are the subfields. However, this can also be derived 
from a more general result of ^GGSZ06i| and |Mat07j . which states that the only 
nonzero additive subgroups of GF(2'"), which contain the inverse of all of their 
nonzero elements, are the subfields. 

Condition (IHl) follows from the properties of the components MixColumns |DR02l 
3.4.3] and ShiftRows [ DRM 3.4.2] of the linear mixing layer (which are not altered 
by the fact that we have incorporated in it the linear part of the S-boxes) . In fact , 
suppose, without loss of generahty, that U ^Vi. Then U' contains the whole first 
column of the state, and U" = V, a. contradiction. This argument is a vestigial 
form of the Four- Round Propagation Theorem [DR02[ 9.5.1]. 

3. Primitivity 
In this section we give a proof of Theorem [TJ 

Suppose for a contradiction that G = (T, p) is imprimitive on V, so that any 
block system for G is given by the cosets of some subspace U of V. This is because, 
as it is proved in jCDVSOS] . a block system for G is also a block system for the 
group T of translations. 

Now p = 7A, with A linear, and O7 = 0. Thus Up = U, and U' = U'-y = U\^^ 
is a subspace. 

Suppose firstly that U = V^^ © ■ ■ ■ © VJ, is a direct sum of some of the subspaces 
Vi {I < Ut). Then, U' = U'y = U, so that U' = U is A-invariant; this contradicts 
Cryptographic Assumption ([H]). 

Thus there exists i such that U ^ Vi, but there is u E U, such that its i- 
th component Ui & Vi is nonzero. We claim that U (1 Vi is nonzero. Take any 
V E Vi. Then {u + f)7 + f 7 G U', so that wj + {u + v)'j + vj E U'. The 
latter element has all zero components, expect possibly the z-th one, which is 
Ui'ji + {ui + v)'ji + V G U' nVi. Were the latter zero for all w G Vi, then the map 
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Vi ^ Vi that maps v ^ {ui + v)^ + vfi would be constant, thus contradicting 
Cryptographic Assumption fl2a|) . 

Thus there exists i such that both Ui = U nVi and f// = {Ui)'~^i = U' (iVi are 
nonzero, proper subspaces of Vi of the same dimension, and 

7i : V,/Ui Vi/Ui 

If X G Vi, and f G t/j, f 7^ 0, then x + v and x are in the same coset of Ui, so 
(x + v)'ji and are in the same coset of f//. Thus the set 

{{x + v)^i + X7i : X G Vi] 

is a subset of t/^', and by Cryptographic Assumption fl2al) [/j and have size 
greater than 2™-"'""^, that is to say dimension at least m — r or equivalently codi- 
mension at most r. The codimension of Ui fl U- is therefore at most 2r, so Ui fl U- 



cannot be 7j-invariant because of Cryptographic Assumption fl2bp . This means 
there exists z G f/j fl U^ such that Z'ji ^ UiD U-, so zji ^ Ui, as Z'ji G However, 
t/j' is the image of Ui under the bijective map 7^, so 2; = Z'jf ^ U^, as Z'ji ^ Ui. 
Thus z ^ UiH U'i, which is a contradiction. 

4. O'Nan-Scott 

In this section we prove Theorem [21 We first state the O'Nan-Scott classifica- 
tion of primitive groups for the case of the maximal primitive subgroups of the 
symmetric group. We give the result for the symmetric group of degree g", where 
g is a power of a prime number p. 

Theorem 3. |Cam99[ Theorem 4.8] Suppose q is a power of the prime p. 
A maximal primitive subgroup G of Sym(g"') is one of the following: 

(1) affine, that is, G = AGL{d,p),p'^ = g", for some d; 

(2) primitive non-basic, that is, a wreath product G = Sym(A;) I Sym(r) in 
product action, k^ = q"',ky^2,r>l. 

(3) almost simple, that is, S < G < Aut(S'), for a nonabelian simple group S. 

Note that in our context p = 2. 

It is convenient to use a refinement of the O'Nan-Scott theorem, due to Cai 
Heng Li |Li03j . for the special case when G contains an abelian regular subgroup 
T; in our case, this is the group of translations. 

Theorem 4. |Li031 Theorem 1.1] Let G be a primitive group of degree 2^^, with 
d > 1. Suppose G contains a regular abelian subgroup T. 
Then G is one of the following 

(1) affine, that is, G < AGL(c/,2); 
(2) 

G= (5i X ■■■ X Sr).O.P, 

with 2'^ = nf for some m and r > 1. Here T = Ti x ■ ■ ■ x T^, with 
Ti < Si = Alt(m) for each i, O < Out(S'i) x • ■ ■ x Out(S'r), and P permutes 
transitively the S^. 

(3) almost simple, that is, S < G < Aut(S'), for a nonabelian simple group S. 
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To prove the first statement of Tlieorem[2]we need to deal witli tlie tliree possible 
cases of Theorem HI 

Case ([T]) is treated in Section An important observation of Li [Li03j is in 
order here. If \^ is a vector space, with addition +, then the symmetric group 
Sym(l^) contains the affine group AGL(\^) = TGL{V), where T is the group of 
translations. But Sym(\^) also contains the conjugates of AGL{V), which are still 
affine groups on the set V, but possibly with respect to an operation o different 
from +. In particular the group T of translations may be contained in one of these 
conjugates, where it will be an abelian regular subgroup. We have studied this 
situation in [CDVS06] . and we will be exploiting these results in Section El 

Case ([2]) will be dealt with in Section [61 

In the almost simple case ([HD, the intersection of a one-point stabilizer in G with 
is a proper subgroup of S of index 2*^, since the nontrivial normal subgroup S of 
the primitive group G is transitive. We can thus appeal (as Li does) to a particular 
case of a result of Guralnick [ Gur83] , which states that the only nonabelian simple 
groups that have a subgroup of index of the form 2"^ are either the alternating 
groups 5* = Alt(2'^), with > 2, or the groups PSL(/, g), where g is a prime- 
power, and / is prime, (g-^ — l)/(g — 1) = 2'^. We rule out the second possibility 
as follows. Since (g-^ — l)/(g — 1) = q^~^ + q^~^ + ■ ■ ■ + q + 1 = f (mod 2), we 
have / = 2 here, and g = 2^^ — 1. Well-known elementary arguments yield that g 
and d are prime. However, d = rttm is not prime, as > 1 by assumption, and 
as noted earlier m > 2 by Cryptographic Assumption ([T|). 

Clearly Aut(Alt(2"')) = Sym(2'^) here, so G is either the alternating or the 
symmetric group. Since we have shown in Section [2l that G < Alt(V^), we obtain 
G = Alt(V^). 

To prove the second statement of Theorem O we then appeal to a standard 
argument: if the nonabelian simple group G is generated by a subset S, then for 
any fixed r the set S' = { S1S2 . . . : Sj G S" } of r-fold products of elements of S 
generates a nontrivial normal subgroup of G, and thus S' also generates G. In our 
context S is the set of the round functions for all possible subkeys, and r is the 
number of rounds, so that S' is the set of the transformations of the cryptosystem 
with independent subkeys. 

5. The affine case 

Suppose G is contained in an affine subgroup of Sym(l^). By the theory 
of [CDVS06] . there is a structure of an associative, commutative, nilpotent ring 
(y, o,-,0) on V, such that (V^, o,0) is a vector space over the field with two ele- 
ments, and ordinary addition on V is expressed as 

x + y = xoyo xy, 

for x,y & V. Moreover, G acts as a group of affine transformations on (V^, o, 0). 
As both {V, o, 0) and (V^, +, 0) are elementary abelian, we have 



O = x + x = xoxoxx = 0ox'^ = x'^ 
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for all X E V. It follows 

X + y + xy = {x o y o xy) o xy o (^x o y o xy) ■ xy 
= xoyoxyoxyo x^y o xy'^ o x^y'^ 
= X oy. 

Here we have used the fact that ■ distributes over o. 

Now p G G is linear with respect to o, that is {xoy)p = xpo yp for all x,y E V. 
Choose y E U = { z E V : xz = for aA\ x E V } . (The latter set is different 
from {0}, as the ring {V, o, 0) is nilpotent.) Then 

(5.1) (x + y)p = {x o y)p = xp o yp = xp + yp + xp ■ yp. 

Now note that given x E V, the set xV = {xz : z E V } is a subspace with 
respect to o, as ■ distributes over o; and also a subspace with respect to +, as 

XZi + XZ2 = XZi O XZ2 O x'^ZiZ2 = XZi O XZ2. 

It follows from 15.11 that for ^ y E U we have 

{ (x + y)p + xp : X E V } = yp + ypV. 

The right hand side is a coset of a subspace of V with respect to +. Now A (and 
its inverse) are linear with respect to +. Applying we obtain that 

{ (x + y)'y + X'j : X E V} 

is also a coset of a subspace of V with respect to +. Choose an index i so that 
the component yi E Vi of y is nonzero. Then we have that the projection on Vi of 
the previous set 

{ (x + yi)j + x-f : X E Vi} 

is a coset of a subspace of Vj with respect to +. This contradicts Cryptographic 
Assumption fl2al) . 

6. Wreath product in product action 

Here we deal to the case when 

G={SiX---X Sr).O.P, 

with 2'^ = for some k and r > 1. Here T = Ti x ■ ■ ■ x T^, where \Ti\ = k 
and Ti < Si = Alt(fc) for each i, O < Out(S'i) x ■ ■ ■ x Out(S',.), and P permutes 
transitively the Si by conjugation. It follows that Si x ■ ■ ■ x Sr = Soc{G). 

Note that if A; = 2 or 4, so that Si = Alt (2) or Alt (4), the group T of translations 
is normal in G, so that G < AGL{V). This contradicts the non-linearity of 7, 
which follows from Cryptographic Assumption (I2aj) . Thus we will assume k > 4 
in the rest of this section. 

Note that G = {T,p), and T < Soc(G'), so that G/ Soc(G) is cyclic, spanned 
by p. Since P permutes transitively the Si, it follows that p permutes cyclically 
the Si by conjugation, that is, we may rename indices so that S'f = p~^Sip = Si+i 
for each i (and indices are taken modulo r). 
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Since each Tj is a group of translations, Wi = OTj C OSi is a subspace of V, of 
order k. Since OSi has also order k, OTj = OSi. Clearly each element of v E V can 
be written uniquely in the form v = Ot, for t eT . Thus 

V = 0tit2 ...tr = 0ti + 0t2-\ hOtr 

for unique ti E Ti, and 

V = WiQ)W2Q)---®Wr. 

For each i we have also Wip = OSip = OS^^^p = OpSj+i = 05*1+1 = W^i+i, as 
Op = 0. Thus p permutes cyclically the Wi. Now let v E V, and write it as 
V = Wi + ■ ■ ■ + Wr where Wi E Wi. Let ti E Wi be such that Wi = Ot,. Since 
the ti are translations, we have v = Oti + 0^2 + ■ ■ ■ + Ot^ = 0tit2 • • • tr- We have 
vp = 0tit2 . . . trP = Otj'ta ■ ■ - f^, as Op~^ = 0. Since tf E S'f = S'j+i, there are t- G T, 
such that Otf = Ot^p = Ot'-_^_^ E Wi^i, and because Si and S'j commute elementwise, 
we have 

vp = Ot?t^ ...tp = ot'^4 ■■■K = ot% ...fp 
= ot'^t'^...tp = ot'^t'^...tp = ... 

= Ota^s ...t[= Ot[ + 0^2 + ■ ■ ■ + 0< 

= Ot^p + Otip H h Otr_ip 

= Wip + W2P H h WrP- 

Now fix an index i, and take u E Wi. We have from the above 

vp = {Wi + W2 -\ h Wr)p = WiP + W2P + H h IfrP, 

where Wip E VFj+i, and also 

{v + U)p = WiP + {Wi + m)p H h WrP 

with (wj + u)p E Wi+i. It follows 

(6.1) (f + u)p + f p = U7ip + {wi + u)p G Wj+i. 

Now p = 7A, where A is linear. Applying A^^ to both sides of (16. ip we get 
{v + m)7 + f 7 G Wi+i\~^. In other words, there are subspaces Wi, Wi+iX~^ of V 
of the same dimension such that when the input difference to 7 is in the first one, 
then the output difference is in second one. By the arguments of Section O (with 
U = Wi and U' = Wi+iX~^), it follows that Wi is the direct sum of some of the Vj, 
for each i. Thus W2 = Wip = WiX and Ws = W2\, contradicting Cryptographic 
Assumption ([3]). 
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